Experiments in socially guided exploration: lessons learned in building robots that learn with and without human teachers

We present a learning system, Socially Guided Exploration, in which a social robot learns new tasks through a combination of self-exploration and social interaction. The system’s motivational drives, along with social scaffolding from a human partner, bias behavior to create learning opportunities for a hierarchical Reinforcement Learning mechanism. The robot is able to learn on its own, but can flexibly take advantage of the guidance of a human teacher. We report the results of an experiment that analyzes what the robot learns on its own as compared to being taught by human subjects. We also analyze the video of these interactions to understand human teaching behavior and the social dynamics of the human-teacher/robot-learner system. With respect to learning performance, human guidance results in a task set that is significantly more focused and efficient at the tasks the human was trying to teach, while self-exploration results in a more diverse set. Analysis of human teaching behavior reveals insights of social coupling between the human teacher and robot learner, different teaching styles, strong consistency in the kinds and frequency of scaffolding acts across teachers, and nuances in the communicative intent behind positive and negative feedback.